Automated HPOMDP Construction through Data-mining Techniques in the Intelligent Environment Domain

Markov models provide a useful representation of system behavioral actions and state observations, but they do not scale well. Utilizing a hierarchy and abstraction as in hierarchical hidden Markov models (HHMMs) improves scalability, but they are usually constructed manually using knowledge engineering techniques. In this paper, we introduce a new method of automatically constructing HHMMs using the output of a sequential data-mining algorithm, Episode Discovery. Repetitive behavioral actions in sensor rich environments can be observed and categorized into periodic episodes through data-mining techniques utilizing the minimum description length principle. From these discovered episodes, we demonstrate an automated technique for creating HHMMs and subsequent hierarchical POMDPs (HPOMDPs) for intelligent environment research. We present the theory of this technique and frame it in a case study involving our MavHome architecture and an on-campus smart apartment with a real